This disclosure generally relates to the display of synchronized side-by-side viewing of camera-captured and computer-generated images of complex objects (such as aircraft) during design, assembly, maintenance, and performance analysis.
When building or maintaining a complex object like an aircraft, it is useful to have access to photographic imagery (e.g., panoramic images) showing the assembled or partially assembled object. But this “as-built” representation takes time to acquire and may not be available for all instances of the target object (especially instances that are not built yet).
A three-dimensional (3-D) model visualization system may be used to display images representing portions and/or individual components of one or more 3-D models of a complex object within a graphical user interface. As used herein, the term “3-D model” means a 3-D computer-aided design (CAD) or other similar model (sometimes referred to as a solid model). Visualization systems may be used to perform various operations with respect to the image of the object. For example, a user may use a 3-D model visualization system to navigate to a location in the 3-D visualization environment to view a particular part or assembly of parts within an object for the purpose of identifying information for use in performing an inspection or a repair. Visualization of 3-D models can provide an “as-designed” representation of the target object or environment and can be created before the target object exists, but changes or updates that occur during assembly might not be available in this representation of the target object.
In the aviation industry, users who need visual information about an aircraft typically walk to the aircraft to see how some portion of the aircraft was assembled or try to determine information from two-dimensional (2-D) drawings. Three-dimensional visualization systems are also available, but do not usually include photorealistic representations or as-built condition of assembly of the scene.
Existing photographic image-based viewer applications, such as “virtual tour” applications that use spherical panoramic images and spherical image viewers, improved this situation by allowing users to view real-world panoramic images of a previous version of the aircraft from multiple locations on a personal computer (PC), laptop, or tablet—but they could not display the current version of the aircraft. As used herein, the term “panoramic image viewer application” refers to the class of photographic image-based viewer applications that use one or more curvilinear or stitched rectilinear photographic images to provide a 360-degree representation of the target environment, a portion of which will be shown to the user in the form of a transformed rectilinear image in the panoramic image viewer application. In addition, an existing 3-D model visualization application also allows users to see models of the nominal designs for specific line number aircraft in context, but the images generated by the 3-D visualization are not usually photorealistic and do not show the physical condition of assembly and may miss some of the subtle features of the actual aircraft.
While existing virtual tour types of applications provide as-built visual data, that data is captured for a specific line number aircraft, which may not be exactly how subsequent (or prior) aircraft look. Also, parts could not be removed from the digital photographs to show what is behind them unless additional photos are captured at different phases of the build. In contrast, the existing 3-D model visualization application has the ability to show 3-D data for each specific line number aircraft, but cannot show the as-built environment, which also has a visual difference (since the images rendered in the 3-D visualization are usually non-photorealistic).
It would be desirable to provide enhancements that address the shortcomings of each of the above-described approaches by providing methods for combined use of data from physical (i.e., photographic or video) images and 3-D models.